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WHAT IS CLAIMED IS: 

1. A makeup simulation program that causes a computer to 
perform a makeup simulation using a reference image comprising 
n (n is a positive integer) layer images and one facial image, 
the reference image having been generated previously according 
to a drawing instruction inputted by a user via a manipulation 
device, each of the n layer images containing at least one makeup 
element image in one color, the makeup simulation program being 
characterized by causing the computer to function as means as 
follows : 

first image generation means for generating n first images 
corresponding to the respective n layer images on the basis of 
a transparency set in each pixel in each layer image for each 
of the n layer images; 

target image acquisition means for acquiring a simulation 
target image containing a face of an individual as a simulation 
target; 

mesh setting means for setting a mesh in a facial region 
contained in the reference image and setting a mesh in a facial 
region contained in the simulation target image; 

deformation means for calculating a difference of 
positions between corresponding vertices of the meshes set in 
the reference image and in the simulation target image, and 
deforming the makeup element image contained in each of the n 
first images to fit in the facial region contained in the 
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simulation target image on the basis of the calculated 
difference; 

second image generation means for generating n second 
images corresponding to the respective n layer images by 
performing a specific color conversion processing using color 
components of the n layer images on the simulation target image; 
and 

synthesis means for determining transparencies of the 
respective n second images and the simulation target image on 
the basis of the n first images, and synthesizing the simulation 
target image and the n second images using the determined 
transparencies . 

2. The makeup simulation program according to Claim 1, 
wherein: 

the first image generation means generates the first 
images by generating transparency layer images on the basis of 
the transparency of each pixel set in the layer images and 
normalizing the transparency of each pixel in the generated 
transparency layer images; and 

the synthesis means synthesizes the simulation target 
image and the second images by alpha blending. 

3. The makeup simulation program according to Claim 2, 
wherein : 



P1510 



the second image generation means generates the second 
images by converting each of a color component of each layer image 
and respective color components of the simulation target image 
from an RGB color system to an HSV color system, performing the 
color conversion processing expressed by Equation (A) through 
Equation (C) below on both the converted color components, and 
converting resulting images from the HSV color system to the RGB 
system: 

H r = H c ... Equation (A) 

S r = 0 . 8 x S f + 0 . 2 x S c ... Equation (B) 

V r = 0.8 x V f + 0.2 x V c ... Equation (C) 
where H c , S c , and V c represent an HSV value of the color component 
of layer image, S f and V f represent the HSV value of each pixel 
in the simulation target image, and H r , S r , and V r represent the 
HSV value of each pixel in the second images. 

4 . The makeup simulation program according to any of Claims 
1 through 3, wherein the mesh setting portion includes: 

initial mesh storage means for storing an initial mesh in 
which all vertices comprise characteristic vertices set at 
characteristic positions of the face and other floating 
vertices; 

reference position detection means for detecting a 
specific reference position from the facial region contained in 
each of the simulation target image and the reference image; 



71 



P1510 



characteristic point extraction means for extracting 
characteristic points corresponding to the characteristic 
vertices from each of the simulation target image and the 
reference image on the basis of the reference position detected 
by the reference position detection means; and 

vertex migration means for moving the characteristic 
vertices to the corresponding characteristic points extracted 
by the characteristic point extraction means, and moving the 
floating vertices while keeping distances among respective 
vertices of the initial mesh constant. 

5. The makeup simulation program according to Claim 4, 
wherein: 

the vertex migration means creates 3-D data for each vertex 
of the initial mesh by providing a height component orthogonal 
to an image plane, and moves the floating vertices by providing 
a certain height component to the floating vertices and 
positioning the characteristic vertices on the image plane while 
applying computations expressed by Equation (D) and Equation (E) 
below on each vertex a predetermined number of times: 
(Mathematical Formula 1) 

Pfpl-Xi | P^Pj - lij)(K-Pj) • • * EQUATION (D) 

PV Pj + 1 ( I Pr P 7l " lip(K-Pp * * * EQUATION (E) 
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where a vector Pj. represents a coordinate of the vertex of the 
mesh, a vector Pj represents a coordinate of the vertex of the 
mesh adjacent to the vector P ± , a vector P'i represents the vector 
Pi to which the computations have been applied, a vector P'j 
represents the vector Pj to which the computations have been 
applied, lij represents a length of a line segment linking the 
vector Pi and the vector Pj, and X represents a constant value. 

6. The makeup simulation program according to Claim 4, 
wherein the reference position detection means detects a 
position between eyebrows in each of the reference image and the 
simulation target image as the reference position, using a left 
eye segment image and a right eye segment image that form an eye 
region image comprising a region containing left and right eyes 
and the position between the eyebrows, and includes: 

correlation value image generation means for generating 
a left eye correlation value image indicating a correlation of 
the left eye segment image with the face, and generating a right 
eye correlation value image indicating a correlation of the right 
eye segment image with the face; 

correlation value image moving means for moving the left 
eye and right eye correlation value images on the basis of a 
relative positional relation of the left eye segment image and 
the right eye segment image with respect to the eye region image, 
so that a region indicating a high correlation within the left 
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eye correlation value image and a region indicating a high 
correlation within the right eye correlation value image overlap 
at the position between the eyebrows; 

correlation value image superimposing means for 
superimposing the left eye and right eye correlation images that 
have been moved by the correlation value image moving means; and 

position-between-eyebrows detection means for detecting 
the position between the eyebrows by detecting a region having 
a correlation value higher than a specific value in the left eye 
and right eye correlation value images superimposed by the 
correlation value image superimposing means. 

7. The makeup simulation program according to Claim 6, 
wherein: 

the correlation value image generation means generates the 
left eye and right eye correlation value images by applying a 
morphology processing on the facial image. 

8. The makeup simulation program according to Claim 7, 
wherein : 

the correlation value image generation means generates the 
left eye and right eye correlation value images by further 
applying a processing to reduce a resolution on the images to 
which the morphology processing has been applied. 
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9. The makeup simulation program according to Claim 6, 
wherein : 

the correlation value image superimposing means 
superimposes the left eye and right eye correlation value images 
by multiplying corresponding pixel data of both the correlation 
value images moved by the correlation value image moving means. 

10. The makeup simulation program according to Claim 9, 
wherein: 

the correlation value image superimposing means 
superimposes the left eye and right eye correlation value images 
by applying a processing to enlarge a region having a high 
correlation on the left eye and right eye correlation value 
images moved by the correlation value image moving means. 

11. The makeup simulation program according to Claim 10, 
wherein: 

the correlation value image superimposing means performs 
a processing to further increase a correlation value in the 
region having the high correlation on the left eye and right eye 
correlation value images moved by the correlation value image 
moving means . 

12. The makeup simulation program according to Claim 1, 
wherein : 
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the simulation target image is each frame image in a moving 

image . 

13. A makeup simulation device that performs a makeup 
simulation using a reference image comprising n (n is a positive 
integer) layer images and one facial image, the reference image 
having been generated previously according to a drawing 
instruction inputted by a user via a manipulation device, each 
of the n layer images containing at least one makeup element image 
in one color, the makeup simulation device being characterized 
by including: 

first image generation means for generating n first images 
corresponding to the respective n layer images on the basis of 
a transparency set in each pixel in each layer image for each 
of the n layer images; 

target image acquisition means for acquiring a simulation 
target image containing a face of an individual as a simulation 
target; 

mesh setting means for setting a mesh in a facial region 
contained in the reference image and setting a mesh in a facial 
region contained in the simulation target image; 

deformation means for calculating a difference of 
positions between corresponding vertices of the meshes set in 
the reference image and in the simulation target image, and 
deforming the makeup element image contained in each of the n 
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first images to fit in the facial region contained in the 
simulation target image on the basis of the calculated 
difference; 

second image generation means for generating n second 
images corresponding to the respective n layer images by 
performing a specific color conversion processing using color 
components of the n layer images on the simulation target image; 
and 

synthesis means for determining transparencies of the 
respective n second images and the simulation target image on 
the basis of the n first images, and synthesizing the simulation 
target image and the n second images using the determined 
transparencies . 

14. A makeup simulation method that causes a computer to 
perform a makeup simulation using a reference image comprising 
n (n is a positive integer) layer images and one facial image, 
the reference image having been generated previously according 
to a drawing instruction inputted by a user via a manipulation 
device, each of the n layer images containing at least one makeup 
element image in one color, the computer including first image 
generation means, target image acquisition means, mesh setting 
means, deformation means, second image generation means, and 
synthesis means, the makeup simulation method being 
characterized by including: 
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a step of generating n first images corresponding to the 
respective n layer images on the basis of a transparency set in 
each pixel in each layer image for each of the n layer images 
by the first image generation means; 

a step of acquiring a simulation target image containing 
a face of an individual as a simulation target by the target image 
acquisition means; 

a step of setting a mesh in a facial region contained in 
the reference image and setting a mesh in a facial region 
contained in the simulation target image by the mesh setting 
means; 

a step of calculating a difference of positions between 
corresponding vertices of the meshes set in the reference image 
and in the simulation target image, and deforming the makeup 
element image contained in each of the n first images to fit in 
the facial region contained in the simulation target image on 
the basis of the calculated difference by the deformation means; 

a step of generating n second images corresponding to the 
respective n layer images by performing a specific color 
conversion processing using color components of the n layer 
images on the simulation target image by the second image 
generation means; and 

a step of determining transparencies of the respective n 
second images and the simulation target image on the basis of 
the n first images, and synthesizing the simulation target image 
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and the n second images using the determined transparencies by 
the synthesis means. 
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